4 research outputs found
On the Usefulness of Synthetic Tabular Data Generation
Despite recent advances in synthetic data generation, the scientific
community still lacks a unified consensus on its usefulness. It is commonly
believed that synthetic data can be used for both data exchange and boosting
machine learning (ML) training. Privacy-preserving synthetic data generation
can accelerate data exchange for downstream tasks, but there is not enough
evidence to show how or why synthetic data can boost ML training. In this
study, we benchmarked ML performance using synthetic tabular data for four use
cases: data sharing, data augmentation, class balancing, and data
summarization. We observed marginal improvements for the balancing use case on
some datasets. However, we conclude that there is not enough evidence to claim
that synthetic tabular data is useful for ML training.Comment: Data-centric Machine Learning Research (DMLR) Workshop at the 40th
International Conference on Machine Learning (ICML
Feature Grouping as a Stochastic Regularizer for High-Dimensional Structured Data
International audienceIn many applications where collecting data is expensive , for example neuroscience or medical imaging, the sample size is typically small compared to the feature dimension. It is challenging in this setting to train expressive, non-linear models without overfitting. These datasets call for intelligent regularization that exploits known structure, such as correlations between the features arising from the measurement device. However, existing structured regularizers need specially crafted solvers, which are difficult to apply to complex models. We propose a new regularizer specifically designed to leverage structure in the data in a way that can be applied efficiently to complex models. Our approach relies on feature grouping, using a fast clustering algorithm inside a stochas-tic gradient descent loop: given a family of feature groupings that capture feature covariations, we randomly select these groups at each iteration. We show that this approach amounts to enforcing a denoising regularizer on the solution. The method is easy to implement in many model archi-tectures, such as fully connected neural networks, and has a linear computational cost. We apply this regularizer to a real-world fMRI dataset and the Olivetti Faces datasets. Experiments on both datasets demonstrate that the proposed approach produces models that generalize better than those trained with conventional regularizers, and also improves convergence speed